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DETAILED ACTION 



Response to Arguments 

1 . Applicant's arguments, see pages 6-12, filed January 17, 2008, with respect to 
the rejections of claims 1 2, 1 6 and 1 8 under 35 USC § 1 02(b) and claims 1 -1 1 and 1 3- 
15 and 17 under 35 USC § 103(a) have been fully considered and are persuasive. 
Therefore, the rejection has been withdrawn. However, upon further consideration, a 
new grounds of rejection is made in view of newly found prior art. 



Double Patenting 

The nonstatutory double patenting rejection is based on a judicially created 
doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the 
unjustified or improper timewise extension of the "right to exclude" granted by a patent 
and to prevent possible harassment by multiple assignees. A nonstatutory 
obviousness-type double patenting rejection is appropriate where the conflicting claims 
are not identical, but at least one examined application claim is not patentably distinct 
from the reference claim(s) because the examined application claim is either anticipated 
by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 
F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 
USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 
1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, All 
F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 
USPQ 644 (CCPA 1969). 

A timely filed terminal disclaimer in compliance with 37 CFR 1 .321 (c) or 1 .321 (d) 
may be used to overcome an actual or provisional rejection based on a nonstatutory 
double patenting ground provided the conflicting application or patent either is shown to 
be commonly owned with this application, or claims an invention made as a result of 
activities undertaken within the scope of a joint research agreement. 

Effective January 1 , 1994, a registered attorney or agent of record may sign a 
terminal disclaimer. A terminal disclaimer signed by the assignee must fully comply with 
37 CFR 3.73(b). 
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2. Claims 15-18 are provisionally rejected on the ground of nonstatutory 
obviousness-type double patenting as being unpatentable over claim 9 of copending 
Application No. 10/685,702 and that of claims 1, 2 and 7 of copending Application No. 
10/685,708. Although the conflicting claims are not identical, they are not patentably 
distinct from each other because: 

Regarding claim 15: Claim 1 of copending Application No. 10/685,708 
recites an information storage medium comprising: AV data; a markup document 
to play the AV data wherein: the markup document includes device-aspect-ratio 
information and is referenced to display the markup document. Claim 15 
discloses a computer readable medium encoded with operating instructions for 
displaying such as the aforementioned markup document comprising: obtaining 
the AV data from the markup document; obtaining the markup picture from the 
markup document which is according (or referenced) to the device-aspect-ratio 
information. 

Regarding claim 16: Claim 2 of copending Application No. 10/685,708 
depends from claim 1 and includes wherein the device-aspect-ratio is referenced 
from a target display device intended to display the markup document. Claim 16 
discloses a computer readable medium encoded with operating instructions for 
processing such as the aforementioned markup document comprising: 
determining whether device-aspect-ratio information from a target display device 
intended to display the markup document is included. 
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Regarding claim 17: Claim7 of copending Application No. 10/685,708 
recites an information storage medium wherein the device-aspect-ratio is 
referenced to transform a source markup picture. Claim 17 discloses 
transforming a source markup picture obtained from the markup document. 

Regarding claim 18: Claim 9 of copending Application No. 10/685,702 
recites an apparatus for processing a markup document in an interactive mode 
comprising: a controller to output a markup picture; and a markup transformer. 
Claim 18 discloses a method in a computer system to process (AV) data in an 
interactive mode using a markup document which consist of the obvious variants 
of the afore mentioned components for the apparatus which comprise: controlling 
an AV playback engine to [output] a video picture; and an (ENAV) engine to 
transform a markup picture. 

This is a provisional obviousness-type double patenting rejection because the 
conflicting claims have not in fact been patented. 

Claim Rejections - 35 USC § 102 

The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 
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3. Claims 1,4,5 and 9 are rejected under 35 U.S.C. 1 02(b) as being anticipated by 
Nielsen (U.S. Pat. No. 5,897,644). 

Regarding claim 1: Claim 1 discloses a method which includes obtaining 
device-aspect-ratio information corresponding to the markup document as well 
as obtaining a markup picture from the markup document according to the 
device-aspect-ratio information. 

While applicant agrees that Nielsen does teach "a method of displaying a 
markup picture according to a size of a display device or window and an aspect 
ratio of the markup picture" which are retrieved from the display device and 
markup document accordingly, applicant states that Nielsen, "teaches a method 
of displaying a markup picture according to a size (but not an aspect ratio) of a 
display device or window." More specifically applicant states that, "Nielsen does 
not disclose a processing of a markup document according to the aspect ratio of 
the display window" and that, "the specification does not disclose nor support an 
obtaining of display device aspect ratio information, let alone an obtaining of 
display device aspect ratio from a markup document." 

However according to the specification (see [col. 7, II. 50-57], [col. 8, II. 20- 
65]) Nielsen does teach obtaining device-aspect-ratio information corresponding 
to the markup document as well as obtaining a markup picture from the markup 
document according to the device-aspect-ratio information from procedures set 
forth in the markup document instructions (see [col. 8, II. 20-36], [col. 8, II. 39-46] 
where Nielsen teaches allocating the entire display device area, bypassing the 
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windowing functions, to ascertain the display device width and height followed by 
obtaining an aspect ratio, where the aspect ratio is calculated as the "floating 
point ratio of Width divided by Height"; see also [col. 8, II. 37-39], [col. 8, II. 50- 
54], [col. 8, II. 61-65] where the markup document is appropriately dimensioned 
in the viewable window based on the aforementioned processes used to 
correlate the determined device-aspect-ratio and the markup document canvas 
specifications, thus obtaining a markup picture according to the device-aspect- 
ratio information). 

Regarding claim 4: Nielsen discloses all of the limitations of claim 1 
including wherein the obtaining of the device-aspect-ratio information comprises 
parsing the device-aspect-ratio information which is written in the markup 
document using a property of a tag (see [col. 7, II. 1-12] where the obtaining the 
canvas specification comprises a parsing mechanism and using unique HTML 
markup tags). 

Regarding claim 5: Nielsen discloses all of the limitation of claim 1 
including wherein the obtainment of the markup picture comprises transforming a 
source markup picture into the markup picture with a predetermined aspect ratio 
according to the device-aspect-ratio information, the source markup picture being 
obtained from the markup document (see [col. 7, II. 66-col. 8, II. 4] where the 
HTML data stream is transformed, according to a predetermined aspect ratio, 
and presented as resultant HTML image; see also [col. 5, II. 7-11] where the 
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HTML data stream can obtain graphics from graphical data; see also [col. 10, II. 
7-9] where other HTML non-textual elements can be transformed). 

Regarding claim 9: Nielsen discloses all of the limitations of claim 1 
including wherein the device-aspect-ratio information is included in the markup 
document (see [col. 3, II. 40-44] where the device-aspect-ratio information, 
referred to herein as the canvas specification, is obtained from within the HTML 
data stream). 



Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. Claims 2-3, 7-8, 1 1 , 12, 13, 16 and 17 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Nielsen (U.S. Pat. No. 5,897,644) in view Nishi (U.S. Patent 
No. 6,681,395 B1). 

Regarding claim 2: Nielsen discloses all of the limitations of claim 1 , 
including wherein the obtaining of the device-aspect-ratio information comprises 
reading information which includes the maximal display size of a target display 
device intended to display the markup document in an interactive mode (see [col. 
7, II. 50-65], [col. 8 , II. 20-65] for the process of accessing information about the 
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parameters of a specific display device intended to present the interactive image 
which includes determining the aspect ratio). 

Nielson does not explicitly teach the device-aspect-ratio information being 
information on an aspect ratio of a target display device intended to display the 
markup document in the interactive mode. 

Nishi teaches a method of modifying the display of a markup document 
based on receiving the aspect ratio and size information of a display screen for a 
viewing device (see [figs. 1, 4, 7, 12 and 13], [col. 6, II. 10-24], [col. 12, II. 24-34]). 

At the time the invention was made, it would have been obvious to one of 
ordinary skill in the art to retrieve the aspect ratio information of the display 
screen for a target viewing device, as taught in Nishi, while accessing the display 
device information to obtain values used to determine the maximal view while 
maintaining the intended aspect ratio, as taught in Nielsen, because by obtaining 
the target display device's aspect ratio, to adjust the display content accordingly, 
it will be possible to support more devices and more quickly eliminate viewing 
distortion caused by non-uniform aspect ratios (see Nishi [col. 1, II. 52-67]). 

Regarding claim 3: Nielsen discloses all of the limitations of claim 1 
including wherein the obtaining the device-aspect-ratio information comprises 
reading information which includes the maximal display size of a target display 
device intended to display the markup document in an interactive mode 
according to a design of a markup document maker (see [col. 7, II. 50-65], [col. 8 
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, II. 20-65]; see also [col. 3, II. 29-37] the HTML image is presented in the original 
proportions as the author intended). 

Nielson does not explicitly teach the device-aspect-ratio information being 
aspect ratio information received directly from a target display device intended to 
display the markup document in the interactive mode. 

Nishi teaches a method of modifying the display of a markup document 
based on receiving, from the device, the aspect ratio and size information of a 
display screen for the target viewing device (see [figs. 1, 4, 7, 12 and 13], [col. 6, 
II. 10-24], [col. 12, II. 24-34]). 

(See claim 2 for the motivation to combine). 

Regarding claim 7: Nielsen discloses all of the limitations of claim 1 
including wherein the obtainment of the markup picture comprises scaling a 
source markup picture to output the markup picture corresponding to device- 
aspect-ratio information in response to information which includes the maximal 
display size and the aspect ratio, the aspect ratio being calculated as the "floating 
point ratio of Width divided by Height", from a destination device being different 
form the calculated device-aspect-ratio information, the destination device 
substantially displaying the markup document and the source markup picture 
being obtained from the markup document (see [col. 8 , II. 20-65]; see also [col. 
7, II. 66 through col. 8, II. 5] where the HTML image can be reduced or magnified 
according to the transformation limitations; see also [col. 8, II. 20-26] for the 
maximization in the device accessible area). 
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Nielson does not explicitly teach the device-aspect-ratio information being 
aspect ratio information received directly from a target display device intended to 
display the markup document in the interactive mode. 

Nishi teaches a method of modifying the display of a markup document 
based on receiving, from the device, the aspect ratio and size information of a 
display screen for the target viewing device (see [figs. 1, 4, 7, 12 and 13], [col. 6, 
II. 10-24], [col. 12, II. 24-34]). 

(See claim 2 for the motivation to combine). 

Regarding claim 8: Nielsen discloses all of the limitations of claim 7, but 
does not explicitly teach wherein the destination device is a television monitor. 

Nishi teaches using a television as a display (see [fig. 1]). 

At the time the invention was made, it would have been obvious to a 
person of ordinary skill in the art to display an HTML data stream image, as 
taught in Nielsen, on a television, as taught in Nishi because televisions are 
commonly used to view media presentations (see [col. 1, II. 18-37]). 

Regarding claim 11: Nielsen discloses all of the limitations of claim 1 
including wherein the obtaining of the device-aspect-ratio information comprises 
reading information which includes the maximal display size of a target display 
device intended to display the markup document and information on a resolution 
of the markup document, the information on the aspect ratio, which is calculated 
as the "floating point ratio of Width divided by Height", and the information on the 
resolution being written in the markup document using a property of a tag (see 
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claim 2 rejection; see also [col. 8 , II. 20-65], [col. 9, II. 58 through col. 10, II. 6] 
where the resolution, number of pixels in the view area, is stored in the markup 
document under the property tag "AREA"). 

Nielson does not explicitly teach the device-aspect-ratio information being 
aspect ratio information received directly from a target display device intended to 
display the markup document in the interactive mode. 

Nishi teaches a method of modifying the display of a markup document 
based on receiving, from the device, the aspect ratio and size information of a 
display screen for the target viewing device (see [figs. 1, 4, 7, 12 and 13], [col. 6, 
II. 10-24], [col. 12, II. 24-34]). 

(See claim 2 for the motivation to combine). 

Regarding claim 12: Nielsen discloses a method of processing a markup 
document in an interactive mode (see [col. 3, II. 38-50]). 

Nielsen further discloses determining whether device-aspect-ratio 
information is included in the markup document (see [col. 3, II. 38-44] for 
detecting aspect ratio of markup document). 

Nielsen discloses display device information, being data including 
information such as size values of the target display device intended to display 
the markup document in the interactive mode (see [col. 3, II. 44-45]; see also 
[col. 7, II. 50-53] for accessing information about the target display device). 

Nielsen teaches processing the markup document according to the display 
device information with respect to the maximal display size information being 
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included in the markup document (see [col. 7, II. 50-57], [col. 8, II. 37-65] where 
the markup application is processed according to the limit allowed by the 
determined transformation values). 

Nielson does not explicitly teach the device-aspect-ratio information being 
aspect ratio information received directly from a target display device intended to 
display the markup document in the interactive mode. 

Nishi teaches a method of modifying the display of a markup document 
based on receiving, from the device, the aspect ratio and size information of a 
display screen for the target viewing device (see [figs. 1, 4, 7, 12 and 13], [col. 6, 
II. 10-24], [col. 12, II. 24-34]). 

(See claim 2 for the motivation to combine). 

Regarding claim 13: Nielsen discloses all of the limitations of 
claim 12 including wherein the processing for the markup document includes 
transforming a source markup picture obtained from the markup document 
according to the device-aspect-ratio information in response to information on an 
aspect ratio, where the device-aspect-ratio is calculated as the "floating point 
ratio of Width divided by Height" obtained from the maximal display size, of a 
destination device displaying the markup document being different from the 
calculated device-aspect-ratio information (see [col. 7, II. 66 through col. 8, II. 5], 
[col. 8, II. 37-65] where the HTML image can be reduced or magnified according 
to the transformation limitations obtained prior to displaying; see also [col. 8, II. 



Application/Control Number: 10/685,701 Page 13 

Art Unit: 2623 

20-26] see also [col. 5, II. 7-11] where the HTML data stream can obtain 
graphics from graphical data). 

Nielsen further discloses outputting the source markup picture once 
transformed in response to the information on the aspect ratio of the destination 
device corresponding to the device's calculated aspect ratio information (see 
[317 and 31 9 of figure 3]). 

At the time of the invention it would be obvious, to one of ordinary skill in 
the art, to display the HTML image "as is" as opposed to transforming its 
dimensions because displaying the HTML image "as is" can be accomplished by 
simply not applying one or more pre-existing processes. 

Nielson does not explicitly teach the device-aspect-ratio information being 
aspect ratio information received directly from a target display device intended to 
display the markup document in the interactive mode. 

Nishi teaches a method of modifying the display of a markup document 
based on receiving, from the device, the aspect ratio and size information of a 
display screen for the target viewing device (see [figs. 1, 4, 7, 12 and 13], [col. 6, 
II. 10-24], [col. 12, II. 24-34]). 

(See claim 2 for the motivation to combine). 

Regarding claim 16: Nielsen discloses a computer readable medium 
encoded with operating instructions for implementing a method of processing a 
markup document in an interactive mode, performed by a computer (see [col. 4, 
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II. 20-29] where a program is embedded on a computer usable medium for the 
expresses purpose of processing a markup HTML data stream). 

Nielsen further discloses determining if the display device information is 
included in the markup document, the display device information being data 
including values on the maximal display size of a target display device intended 
to display the markup document in the interactive mode (see [col. 4, II. 20-29]; 
see also [col. 7, II. 50-65], [col. 8, II. 20-65] where the process of accessing 
information about a specific display device, intended to present the interactive 
capable markup data, includes retrieving values which relate to the maximal size 
allowed by the display device). 

Nielsen further discloses processing the markup document according to 
the maximal display size information in response to the maximal display size 
information being included in the markup document (see [col. 4, II. 20-29]; see 
also [col. 7, II. 50-57] where the markup application is processed according to 
the limit allowed by the determined maximal display size values). 

Nielson does not explicitly teach the device-aspect-ratio information being 
aspect ratio information received directly from a target display device intended to 
display the markup document in the interactive mode. 

Nishi teaches a method of modifying the display of a markup document 
based on receiving, from the device, the aspect ratio and size information of a 
display screen for the target viewing device (see [figs. 1, 4, 7, 12 and 13], [col. 6, 
II. 10-24], [col. 12, II. 24-34]). 
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(See claim 2 for the motivation to combine). 

Regarding claim 17: Nielsen discloses all of the limitations of claim 16 
including wherein the processing of the markup document includes transforming 
a source markup picture obtained from the markup document according to the 
device-aspect-ratio information contained in the markup document, in response 
to information on a calculated aspect ratio of a destination display device used to 
display the markup document being different from the device-aspect-ratio 
information contained in the markup document; where the destination device's 
aspect ratio is calculated as the "floating point ratio of Width divided by Height" 
obtained from the maximal display size. 

Nielsen further discloses outputting the source markup picture once 
transformed in response to the information on the calculated aspect ratio of the 
destination device corresponding to the device-aspect-ratio information (see [317 
and 319 of figure 3]). 

At the time of the invention it would be obvious, to one of ordinary skill in 
the art, to display the HTML image "as is" as opposed to transforming its 
dimensions because displaying the HTML image "as is" can be accomplished by 
simply not applying one or more pre-existing processes. 

Nielson does not explicitly teach the device-aspect-ratio information being 
aspect ratio information received directly from a target display device intended to 
display the markup document in the interactive mode. 
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Nishi teaches a method of modifying the display of a markup document 
based on receiving, from the device, the aspect ratio and size information of a 
display screen for the target viewing device (see [figs. 1, 4, 7, 12 and 13], [col. 6, 
II. 10-24], [col. 12, II. 24-34]). 

(See claim 2 for the motivation to combine). 
5. Claim 6 is rejected under 35 U.S.C. 103(a) as being unpatentable over Nielsen 
(U.S. Pat. No. 5,897,644) in view of Yamauchi et al. (U.S. Pat. No. 5,907,659) and Nishi 
(U.S. Patent No. 6,681 ,395 B1). 

Regarding claim 6: Nielsen discloses all of the limitations of claim 1 , 
including wherein the obtainment of the markup picture comprises transforming a 
source markup picture into some markup picture according to device-aspect-ratio 
information, calculated as the "floating point ratio of Width divided by Height", the 
source markup picture being obtained from the markup document (see claim 5 
rejection). 

Nielsen does not explicitly teach transforming said source markup picture 
into a markup picture with an aspect ratio of 4:3 or 16:9. 

Yamauchi teaches the transformation to aspect ratios of 4:3 and 16:9 (see 
[col. 2, II. 19-49]). 

At the time the invention was made it would have been obvious, to a 
person of ordinary skill in the art, to transform an HTML data stream image, as 
taught in Nielsen, into an image with an aspect ratio of 4:3 or 16:9 because these 
aspect ratio standards are widely accepted and used. 
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Nielson does not explicitly teach the device-aspect-ratio information being 
aspect ratio information received directly from a target display device intended to 
display the markup document in the interactive mode. 

Nishi teaches a method of modifying the display of a markup document 
based on receiving, from the device, the aspect ratio and size information of a 
display screen for the target viewing device (see [figs. 1, 4, 7, 12 and 13], [col. 6, 
II. 10-24], [col. 12, II. 24-34]). 

(See claim 2 for the motivation to combine). 
6. Claim 10, 14-15 and 18 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Nielsen (U.S. Patent No. 5,897,644) in view of Betz et al. (Pre Grant 
Pub. 2002/01 061 91 A1 ) and Nishi (U.S. Patent No. 6,681 ,395 B1 ). 

Regarding claim 10: Nielsen discloses all of the limitations of claim 1 
including wherein the displaying of the interactive picture further includes 
obtaining embedding information included in the markup document to combine 
the graphic picture with respect to the markup picture (see [col. 3, II. 40-44] 
where the information is obtained from within the HTML data stream; see also 
[col. 5, II. 7-11] where the HTML data stream can contain graphics obtained 
through processing graphical data; see also [col. 10, II. 7-9] where other HTML 
non-textual elements which include dimensions or parameters are transformed 
prior to being displayed). 
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Nielsen does not explicitly disclose a method of processing a source 
markup picture of a markup document and a source audio video (AV) picture of 
an AV data in an interactive mode. 

Betz teaches processing and compiling video segments which can include 
audio from digital video using a software layer (see [0008], [0009], [0010], [0034], 
[0038]). 

At the time the invention was made, it would have been obvious to one of 
ordinary skill in the art to apply the methods used to provide for a fixed canvas 
presentation to maintain aspect ratio, as taught in Nielsen, to process AV data, 
as taught in Betz, because similarly to graphic pictures, AV data which also 
contains dimensions and parameters must be transformed to be displayed in a 
uniform dimension, more specifically its aspect ratio, on differing size computer 
displays (see Nielsen [col. 10, II. 7-9]). 

Regarding claim 14: Nielsen discloses obtaining a graphic picture typed 
selection information (see [col. 9, II. 45-57] where the graphic or referenced 
media type is selected based on the HTML element tag; see also [col. 6, II. 26- 
40]); determining whether the maximal display size information is included in the 
markup document (see [col. 3, II. 38-44], [col. 8 , II. 37-46] for detecting the 
maximal size of markup document); transforming the source graphic picture and 
the source markup picture according to the graphic picture type selection 
information and the device-aspect-ratio information which is calculated as the 
"floating point ratio of Width divided by Height", respectively, in response to the 
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maximal display size information being included in the markup document (col. 3, 
II. 38-50], [col. 8 , II. 20-65] for transformation; see also [col. 7, II. 50-53] for 
obtaining the maximal display size information). 

Nielsen further discloses displaying the canvas which can include a 
graphic picture according to the graphic picture type selection information and 
outputting the source markup picture "as is" in response to the maximal display 
size information not being included in the markup document (see [fig. 3 item 
sequence 301 , 303, 305, 307, 309], [col. 6-7, II. 66-10] where the canvas is 
displayed without a transformation, "as is" if no specification information is 
present). 

Nielsen does not explicitly disclose a method of processing a source 
markup picture of a markup document and a source audio video (AV) picture of 
an AV data in an interactive mode. 

Betz teaches processing and compiling video segments which can include 
audio from digital video using a software layer (see [0008], [0009], [0010], [0034], 
[0038]). 

At the time the invention was made, it would have been obvious to one of 
ordinary skill in the art to apply the methods used to provide for a fixed canvas 
presentation to maintain aspect ratio, as taught in Nielsen, to process AV data, 
as taught in Betz, because similarly to graphic pictures, AV data which also 
contains dimensions and parameters must be transformed to be displayed in a 



Application/Control Number: 10/685,701 Page 20 

Art Unit: 2623 

uniform dimension, more specifically its aspect ratio, on differing size computer 
displays (see Nielsen [col. 10, II. 7-9]). 

Nielson does not explicitly teach the device-aspect-ratio information being 
aspect ratio information received directly from a target display device intended to 
display the markup document in the interactive mode. 

Nishi teaches a method of modifying the display of a markup document 
based on receiving, from the device, the aspect ratio and size information of a 
display screen for the target viewing device (see [figs. 1, 4, 7, 12 and 13], [col. 6, 
II. 10-24], [col. 12, II. 24-34]). 

(See claim 2 for the motivation to combine). 

Regarding claim 15: Nielsen discloses a computer readable medium 
encoded with operation instructions for implementing a method of displaying an 
interactive picture using a markup document, performed by a computer (see [col. 
4, II. 20-29]). 

Nielsen further discloses obtaining maximal display size information 
corresponding to the markup document (see [col. 3, II. 38-44] contents of markup 
document obtained and used). 

Nielsen further discloses obtaining a markup picture from the markup 
document according to the device-aspect-ratio, calculated as the "floating point 
ratio of Width divided by Height" (see [col. 7, II. 66-col. 8, II. 4], [col. 8 , II. 37-65] 
where the HTML data stream is transformed, according to a predetermined 
aspect ratio, using a presentation mechanism to present a resultant HTML 
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image; see also [col. 5, II. 7-11] where the HTML data stream can obtain 
graphics from graphical data; see also [col. 10, II. 7-9] where likewise other 
HTML non-textual elements can be obtained and transformed). 

However, Nielsen does not explicitly teach a computer readable medium 
encoded with operation instructions for implementing a method of displaying an 
interactive picture using audio video (AV) data, performed by a computer 
comprising obtaining an AV picture from the AV data and displaying the 
interactive picture including combining the AV picture and the markup picture. 

Betz teaches presenting a video or image montage by referencing 
markings which link titles on a digital video or DVD which contain individual 
shots, scenes, or sequences (see [0008-0009] and [0029]). Betz further teaches 
that the aforementioned media can contain audio as well as video (AV) data (see 
[0034], [0038]). 

At the time the invention was made it would have been obvious to a 
person of ordinary skill in the art to obtain and combine AV elements, as taught in 
Betz, for the use and integration with the HTML markup processing, as taught in 
Nielsen, because the AV elements like the graphical elements in Nielsen are 
common media elements and often used in HTML markups (see Betz [0008]). 

Nielson does not explicitly teach the device-aspect-ratio information being 
aspect ratio information received directly from a target display device intended to 
display the markup document in the interactive mode. 
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Nishi teaches a method of modifying the display of a markup document 
based on receiving, from the device, the aspect ratio and size information of a 
display screen for the target viewing device (see [figs. 1, 4, 7, 12 and 13], [col. 6, 
II. 10-24], [col. 12, II. 24-34]). 

(See claim 2 for the motivation to combine). 

Regarding claim 18: Nielsen discloses a method in a computer system to 
process graphic picture data in an interactive mode using a markup document, 
the method comprising: controlling an graphic picture display engine to obtain a 
graphic picture of the graphic data; and controlling an graphic picture display 
engine to transform a markup picture of the markup document according to 
device-aspect-ratio information included in the markup document corresponding 
to the calculated aspect ratio of the target display device intended to display the 
markup document in the interactive mode (see [col. 3, II. 38-50] for a computer 
controlled method; see also [col. 7, II. 66 through col. 8, II. 5] where method is 
capable of transforming HTML data into an HTML image; see also [col. 7, II. 50- 
53] where method is able to obtain the maximal display size of the target display; 
see also [col. 6, II. 26-40] for a description of a general computer which is 
capable of processing multiple forms of image data). 

Nielsen does not explicitly disclose a method of processing a source 
markup picture of a markup document and a source audio video (AV) picture of 
an AV data in an interactive mode or enhanced audio video. 
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Betz teaches processing and compiling of AV and enhanced AV from 
digital video using a software layer (see [0008], [0009], [0010], [0034], [0038]; 
see also [abstract], [0008], [0009], [0012] for enhanced AV through the means of 
presenting as a montage). 

At the time the invention was made, it would have been obvious to one of 
ordinary skill in the art to apply the methods used to provide for a fixed canvas 
presentation to maintain aspect ratio, as taught in Nielsen, to process AV data, 
as taught in Betz, because similarly to graphic pictures, AV data which also 
contains dimensions and parameters must be transformed to be displayed in a 
uniform dimension, more specifically its aspect ratio, on differing size computer 
displays (see Nielsen [col. 10, II. 7-9]). 

Nielson does not explicitly teach the device-aspect-ratio information being 
aspect ratio information received directly from a target display device intended to 
display the markup document in the interactive mode. 

Nishi teaches a method of modifying the display of a markup document 
based on receiving, from the device, the aspect ratio and size information of a 
display screen for the target viewing device (see [figs. 1, 4, 7, 12 and 13], [col. 6, 
II. 10-24], [col. 12, II. 24-34]). 

(See claim 2 for the motivation to combine). 
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Conclusion 

Because new grounds of rejections have been made to reject claims 1-18, this 
Office Action is made Non-Final. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jason Thomas whose telephone number is (571) 270- 
5080. The examiner can normally be reached on Mon. - Thurs., 8:00a.m. - 5:00p.m. 
EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Dennis Chow can be reached on (571) 272-7767. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

J. Thomas /Son P Huynh/ 

Primary Examiner, Art Unit 2623 
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